There is an information system that acquires compressed files accumulated in plural data sources and executes data processing of combining, arithmetic operation, and so forth.
FIG. 20 is a diagram for explaining one example of an information system of a related art. The system illustrated in FIG. 20 includes data sources 10a and 10b and extraction, transmission, and loading (ETL) 20. The ETL 20 is coupled to the data sources 10a and 10b. The ETL 20 may be coupled to another data source that is not diagrammatically represented.
The data source 10a compresses a comma separated values (CSV) file F1 based on the ZIP and accumulates a compressed file F1′ resulting from the compression in a storing unit. The data source 10b compresses a CSV file F2 based on the ZIP and accumulates a compressed file F2′ resulting from the compression in the storing unit. The data source 10a transmits the compressed file F1′ to the ETL 20 in response to an acquisition request from the ETL 20. The data source 10b transmits the compressed file F2′ to the ETL 20 in response to an acquisition request from the ETL 20.
The ETL 20 acquires the compressed files F1′ and F2′ by transmitting the acquisition requests of the compressed files F1′ and F2′ to the data sources 10a and 10b. The ETL 20 executes data processing of combining, arithmetic operation, and so forth for the compressed files F1′ and F2′ and generates a CSV file F3 as the arithmetic operation result. One example of processing of the ETL 20 will be described below.
The compressed files F1′ and F2′ are not compressed in units of comma (or column). For this reason, after obtaining the CSV files F1 and F2 by decompressing the compressed files F1′ and F2′, the ETL 20 carries out lexical analysis and executes the data processing of combining, arithmetic operation, and so forth. In the example illustrated in FIG. 20, the ETL 20 executes arithmetic operation of the total amount of each product code based on master data M1.
For example, when the ETL 20 combines the CSV files F1 and F2, the quantity becomes “20” in total regarding a product code “1242” and a product name “Candy.” Because the unit price is “50,” the total amount becomes “1000 (20×50).” Similarly, regarding a product code “2342” and a product name “Chocolate,” the quantity becomes “30” in total and the unit price is “500,” and therefore the total amount becomes “15000 (30×500).”
As described above, the ETL 20 generates the CSV file F3 by executing the processing of combining, arithmetic operation, and so forth after decompressing the compressed files F1′ and F2′ and carrying out the lexical analysis. Then, the ETL 20 obtains a compressed file F3′ by compressing the CSV file F3 based on the ZIP. The ETL 20 transmits the compressed file F3′ to an external apparatus that is not diagrammatically represented and is the request source of the data processing.
For example, related arts are disclosed in Japanese Laid-open Patent Publication No. 2011-145883 and Japanese Laid-open Patent Publication No. 2011-090526.